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We implement a complete randomized benchmarking protocol on a system of two superconducting 
qubits. The protocol consists of randomizing over gates in the Clifford group, which experimen- 
tally are generated via an improved two-qubit cross-resonance gate implementation and single-qubit 
unitaries. From this we extract an optimal average error per Clifford of 0.0936. We also perform 
an interleaved experiment, alternating our optimal two-qubit gate with random two-qubit Clifford 
gates, to obtain a two-qubit gate error of 0.0653. We compare these values with a two-qubit gate 
error of ~ 0.12 obtained from quantum process tomography, which is likely limited by state prepa- 
ration and measurement errors. 



PACS numbers: 03.67.Ac, 42.50.Pq, 85.25.-j 

A task of fundamental importance in the development 
of large-scale quantum processors is that of determining 
a robust, multi-platform standard to quantify the qual- 
ity of the quantum operations. Characterizing quantum 
gates becomes harder as the error of the gates continues 
to decrease. Quantum process tomography [1, 2], a stan- 
dard method for characterizing quantum gates, gives a 
full description of the protocol under study, but is very 
sensitive to state preparation and measurement (SPAM) 
errors, especially when the errors involved are compara- 
ble to those of the single qubit unitaries of the system. As 
an alternative, recent work has been devoted to the de- 
velopment of randomized benchmarking (RB) protocols 
[3-5], which have been implemented in ion traps [4, 6, 7], 
NMR [8], superconducting qubits [9-11] and atoms in 
optical lattices [12]. Although a RB protocol offers a re- 
liable estimate of the average error per operator within 
a group of operators in its original conception, it does 
not provide a complete description of a given quantum 
process. However, recent studies [13, 14] have presented 
a different RB implementation in which the error of a 
particular gate is measured by interleaving it between 
other random gates within a sequence. Furthermore, RB 
has also been used to determine addressability errors and 
correlations in many-body quantum systems [15]. 

In this Letter, we present a complete RB characteriza- 
tion of two fixed- frequency superconducting qubits. An 
important feature of a RB protocol is the long sequences 
required to implement it. This sets additional constraints 
on the chosen gates, which have to be robust as a part 
of a long series of pulses in the same way they would in 
a real quantum algorithm. In our case, the realization 
of the RB protocol is made possible by modifying a pre- 
viously developed cross-resonance (CR) two-qubit gate 
[16, 17] through refocusing the single-qubit dynamics to 
simplify the gate calibration. 

Our implementation of the RB protocol is restricted to 
the Clifford group C of operators, which is the normalizer 



of the Pauli group V -that is, for each C G C, CPC ] G V, 
with P G V . Having the Clifford group as our set of op- 
erators is justified by two main reasons. First, a Clifford- 
based RB protocol is more readily extendable to systems 
with higher number of qubits, as choosing a random Clif- 
ford and decomposing it into a set of generators (elemen- 
tary qubit operations) scales polynomially in the number 
of qubits [5, 18]. Second, as the set of generators tends to 
vary across systems, such a protocol offers portability be- 
tween the different physical implementations of quantum 
processors. 

For two-qubit systems, the Clifford group is gener- 
ated from single-qubit unitaries and a controlled-NOT 
(CNOT) gate. When implementing the protocol, it is im- 
portant to use a Clifford decomposition into elementary 
unitaries that minimizes the number of average two-qubit 
gates per Clifford, as these tend to have lower fidelities 
than single-qubit gates, especially in the case of super- 
conducting qubits. Rather than following the procedure 
outlined in Ref. [5] for generating a random Clifford oper- 
ation, here we use an optimized set of Clifford operations 
from which we randomly select elements. We divided the 
two-qubit Clifford group into four classes [27]: a class 
with 576 elements containing only single-qubit unitaries 
from the group {I,X± n /2,Y± n /2,X ir ,Y jr }, where Uq rep- 
resents a rotation of angle around the axis £7, a class 
with 5184 elements containing single-qubit unitaries and 
one CNOT gate, a class with 5184 elements containing 
single-qubit unitaries and two CNOT gates and a class 
with 576 elements containing single-qubit unitaries and a 
SWAP gate, implemented by three CNOT gates. There- 
fore, the total number of elements in the two-qubit Clif- 
ford group is 11520, with an average of 1.5 CNOT gates 
per Clifford [28]. 

The experiments are performed on two single-junction 
transmons coupled via a superconducting coplanar 
waveguide resonator, which is also used for readout [19]. 
The qubit resonance frequencies are uj\j2i\ = 3.2324 and 
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FIG. 1. (color online) Homodyne signal as a function of the 
cross-resonance tone length and pulse sequences for the stan- 
dard cross-resonance gate (a) and for the ZJ_^/ 2 (b). All 
tones are applied to the control qubit driving line (Ql), with 
the 7r pulses and cross-resonance tones applied at the con- 
trol and target qubit frequencies, respectively. The n pulse 
between the two cross-resonance tones in the ZX-^/2 gate, 
along with the opposite sign of the second cross-resonance 
pulse, echo away the fast IX component of the driving Hamil- 
tonian seen in (a) and leave only the ZX interaction. The 
dashed vertical lines mark the gate length used for the RB 
experiments for the ZX_ 7r / 2 gate and the equivalent length 
for the standard cross-resonance gate. 

CjJ 2 /2tt = 3.2945 GHz, with anharmonicities 5\ = (uj\ 2 — 
uj^)/2tt = -331 and S 2 = (u 2 2 - oof)/2ii = -216 MHz, 
whereas the bare resonator frequency is uj y /2k = 8.2855 
GHz. The energy relaxation times of both qubit s are 
observed to be — 11.6 /is and =9.1 /is, and 
the coherence times observed from Ramsey-fringe exper- 
iments are — 7.1 /is and = 5.6 /is [27]. The 
qubits are thermally anchored to the coldest stage of a 
dilution refrigerator with a nominal base temperature of 
15 mK and are carefully shielded against thermal radia- 
tion [20, 21]. 

Implementing the Clifford group of operators in our ex- 
periments required modifying a previously demonstrated 
two-qubit CR gate [16]. The basis of the original CR gate 
[22, 23] involves the driving of the control qubit at the 
frequency of the target qubit. This results in a driving 
Hamiltonian of the form 

H D /h « e{t) (mIX - fiZX + rjZI^j (1) 

where {/, X, F, Z}® 2 are the two-qubit Pauli operators, 
e is the drive amplitude, \i is a coupling parameter that 
equals J/ A for ideal qubits, where J is the qubit-qubit 
coupling energy and A is the frequency detuning between 
the qubits, m is a scalar representing the effect of spu- 



rious electromagnetic crosstalk between both qubits as 
well as the effect of higher energy levels, and r] repre- 
sents the magnitude of the Stark shift arising from the 
off-resonant driving of qubit 1. The term mIX results in 
Rabi-like oscillations of qubit 2, to which the term —fiZX 
contributes with a slower rotation whose sign depends on 
the state of qubit 1. The effect of the Hamiltonian Hd 
can be seen in the experiments shown in Fig. 1 (a). A CR 
flattop Gaussian pulse of variable length t\ and Gaussian 
width a = 8 ns is applied to qubit 1 (control) at the fre- 
quency of qubit 2 (target). Depending on whether a 7r 
rotation is applied to qubit 1 prior to the CR pulse (cir- 
cles) or not (triangles), different Rabi rates are observed 
on qubit 2. In the experiments where qubit 1 is in the 
excited state prior to the CR pulse, an additional ir ro- 
tation is applied to qubit 1 at the end of the sequence in 
order to have the Rabi-oscillation signal from the joint 
readout between the same two computational states 1 00) 
and 1 01). All single-qubit rotations are Gaussian-shaped 
pulses with a total length of 32 ns and a = 8 ns. To avoid 
leakage into higher energy levels, all single-qubit pulses 
include a calibrated derivative in the other quadrature 
[9, 24]. 

The Hamiltonian in Equation 1, however, presents dif- 
ficulties for implementing long sequences of Cliffords due 
to the single-qubit terms. These terms could be explic- 
itly tuned out with additional simultaneous pulses, but 
this would be rather demanding on the phase-locking and 
amplitude stability requirements of the electronics and 
on sequence complexity which would have an important 
negative impact on the measured gate fidelity. Instead, 
we construct a more manageable two-qubit Clifford by 
modifying the original pulse sequence in order to remove 
all terms except the one corresponding to ZX. The new 
pulse sequence divides the CR pulse in three parts: an 
initial CR pulse of duration t 2 , a tt rotation of qubit 1, 
and a final CR pulse of opposite sign to the first one, also 
of duration r 2 . As in the original pulse scheme, a final 
7r rotation is applied to qubit 1 when pertinent. The net 
effect of the new pulse sequence is to effectively "echo" 
away the terms involving IX and ZI in Hp and, as a 
result, only the slower Rabi-rotation arising from ZX 
is observed [Fig. 1 (b)]. In the experiments presented 
here, the gate is realized by chosing a value of r 2 that 
leaves qubit 2 in a superposition of |0) and |1). We will 
therefore call this gate ZX_^j 2l the minus sign arising 
from the negative term involving ZX in Equation 1. The 
dashed line in Fig. 1 (b) at r 2 = 178 ns shows the du- 
ration of each of the CR pulses for the ZX_ 7r / 2 gate for 
these data. The total ZX^j 2 gate length is, therefore, 
2 x r 2 + 2 x 32 = 420 ns. The equivalent gate length for 
the original CR scheme is likewise shown in Fig. 1 (a) at 
r\ = 356 ns. 

We performed quantum process tomography on the 
178 ns ZX_ 7r / 2 gate (Fig. 2) by preparing an overcom- 
plete set of 36 states generated by {/, X n , X± 7V / 2 , Y± 7r / 2 }, 



3 



1 0.8 0.6 0.4 0.2 -0.2-0.4-0.6-0.8 -1 



a) 



3.676 us 



(a) ii 

IX 
IY 

.2 iz 
2 xi 

& Yl 

o zi 

- XX 

a XY 

Q. XZ 

D YX 

a. YY 

£ YZ 

zx 

ZY 
ZZ 



(b)l 



II IX IY IZ XI Yl ZIXXXYXZYXYYYZZXZYZZ II IX IY IZ XI Yl ZIXXXYXZYXYYYZZXZYZZ 



Output Pauli Operator 



Output Pauli Operator 



applying the gate to each of them and performing state 
tomography. We use the Pauli basis to represent quan- 
tum process tomography through the Pauli transfer ma- 
trix IZ [17]. The raw data is post-processed with a 
semidefinite algorithm [17] to take into account physi- 
cality constraints such as complete positivity and trace 
preservation of the process. We obtain a gate fidelity 
from the raw data of F g = 0.8830 and a maximum- 
likelihood estimated fidelity of F m i e = 0.8799. These 
results, as the RB measurements described next suggest, 
are probably dominated by SPAM errors. 

We base our implementation of a two-qubit RB pro- 
tocol on the theory described in Refs. [5] and [14]. 
We randomly choose a sequence {Ci, C2, C20} of 20 
Cliffords amongst the 11520 elements of the two-qubit 
Clifford group. From this sequence, we then construct 
a series of truncations {mi, 777,2, ^20}, where rrii = 
{Ci, C2, Ci}. In order to make each truncation self- 
inverting, a final pulse Ii is appended at the end of each 
of them which returns the system to the 1 00) state. The 
state of the two qubits is then read out. A whole RB 
experiment, therefore, consists of a series of pulse trains 
{JVi,JV 2 ,...,JV 20 }, where N { = {ra^ii}. 

The fidelity of the 1 00) state after each train of pulses 
iVi, obtained by joint readout of the two qubits [25], can 
be fit to an exponential model F(i, 1 00) ) = Aa l + £?, and 
the average error rate per Clifford is related to a by the 
relation r = l — a — (1 — a)/d, where d = 2 n for n qubits 
[5]. In this model, state preparation and measurement 
errors are absorbed by the constants A and B and there- 
fore the parameter a provides a SPAM-free estimation of 
the average error per Clifford. One of the shortcomings 
of this model as described, however, is that it only gives 
an estimation of the average error per Clifford generator. 

We can, nonetheless, modify the protocol in order to 
estimate the error of a particular gate of interest, C, also 
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FIG. 2. (color online) Quantum process tomography of the 
ZX_ 7r / 2 gate with T2 = 178 ns. (a) Experimentally extracted ^" 45~ 
Pauli transfer matrix. The gate fidelity is F g = 0.8830 raw 
and F m ie = 0.8799 after applying a maximum likelihood al- 
gorithm, (b) Ideal Pauli transfer matrix representation of the 
ZX-ir/2 gate. 
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FIG. 3. (color online) RB experiments on a two-qubit sys- 
tem, (a) Two-qubit pulse sequence for 5 Cliffords randomly 
selected from the 11520 elements of the two-qubit Clifford 
group. Tall Gaussians represent tt rotations, whereas short 
ones represent ty/2 rotations. A final Clifford is added to 
make the sequence self-inverting, (b) Fidelity decay for the 
1 00) in the standard RB protocol where the twirling is per- 
formed over the two-qubit Clifford group (circles) and in the 
interleaved protocol (triangles) , in which each Clifford has an 
additional ZX_ 7r / 2 gate appended at the end. The decays 
are fitted to an exponential model to extract an average error 
per Clifford of r = 0.0936 ± 0.0058 (standard protocol) and 
a error of r c = 0.0653 ± 0.0014. The arrow shows 

the truncation that would correspond to the pulse sequence 
shown in (a) and which has an average 1 00) state fidelity of 
over 50%. 



belonging to the Clifford group, as described in Refs. [13] 
and [14]. In the new protocol, we interleave random ele- 
ments of the Clifford group between the gate under study 
C. This is achieved by constructing similar random se- 
quences of Cliffords as in the original implementation and 
then appending the gate C after each element in the se- 
quence. A final inverting Clifford is added at the end of 
each truncation of the sequences. 

Fig. 3 (b) shows the average 1 00) state fidelity over 
random sequences of Cliffords with lengths ranging from 
1 to 20 for both the standard (circles) and the interleaved 
(triangles) RB protocols. Each data point was averaged 
over 40 different random sequences. Both sets of RB 
data in Fig. 3 (b) are well described by an exponen- 
tial model, with reduced \ 2 values of 0.589 and 0.386 for 
the standard and interleaved RB protocols, respectively. 
Our construction of the Clifford group results in 30 two- 
qubit gates on average for a 20 Clifford long sequence. 
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The pulse sequence of one particular self- inverting 5- 
Clifford long sequence, comprising 7 two-qubit and 23 
single-qubit gates, is shown in Fig. 3 (a). In the case 
of the standard RB implentation experiment, we obtain 
a = 0.8752 ±0.0078, which results in an average error per 
Clifford of r = 0.0936±0.0058. For the interleaved exper- 
iment, ac = 0.7990 ± 0.0058, from which tq can be esti- 
mated as r c = (d-i)(l-otc/ot)/d = 0.0653±0.0014 [13]. 
All errors here represent a la confidence interval obtained 
from the Jacobian of the fitting model [15]. The aver- 
age error per Clifford for the single-qubit Clifford group 
for each qubit was also measured and observed to be 
n = 0.0041 ±0.0001 for qubit 1 and r 2 = 0.0048 ±0.0002 
for qubit 2 [27]. 

The effective coupling strength of the two qubit s, given 
by the product e(t)/i multiplying the ZX interaction in 
Eq. 1, can be increased with the amplitude of the CR 
driving tone e(t). Thus, larger driving amplitudes result 
in a faster evolution of the system and, therefore, faster 
oscillations in Fig. 1 (b) and shorter T2. The ultimate 
limit for the speed of the ZX_ n / 2 gate is determined 
by the frequency of the oscillations induced by the ZX 
term, the qubit-qubit detuning A, and the qubit anhar- 
monicities. For the strongest drives, energy leakage into 
other levels in the system prevent the gate from becoming 
faster [16, 26]. 

We applied the two-qubit RB protocol to our system 
for different ZX_ 7r / 2 lengths, with r 2 ranging from 115 
to 800 ns. Fig. 4 shows the average error per Clifford 
as a function of r 2 and the calculated coherence-limited 
average error for the measured T 2 of both qubits (dashed 
line) and for T 2 (1 ' 2) = 2T 1 (1 ' 2) (dash-dotted line). We see 
that the experimentally extracted average errors fall in- 
side the shaded region. For the two shortest realizations 
of the ZX_ 7r / 2 , however, the RB experiments yielded an 
average error per Clifford worse than the limit imposed 
by coherence times. This was probably due to leakage 
into higher qubit levels at high CR driving amplitudes 
and to the spurious off-resonance driving of the control 
qubit. For longer gates, the observed experimental values 
indicate that our ZX_ 7r / 2 is essentially coherence limited. 
We attribute the scattering of the data points in Fig. 4 
to small variations in T\ and T 2 , which were observed to 
move by about 1 /is up or down for both qubits over the 
interval of six to twelve hours, approximately equal to 
the amount of time taken to perform the RB protocol for 
each ZX_^j2 gate length. 

In conclusion, we have implemented a RB protocol on 
two superconducting qubits by using a new implemen- 
tation of an entangling two-qubit gate plus single-qubit 
unitaries. The new gate pulse sequence echoes away the 
terms in the gate Hamiltonian involving single-qubit ef- 
fects, which tend to be non-reproducible in long gate se- 
quences. The new gate implementation allows the exper- 
imental realization of the two-qubit RB protocol for the 
first time in superconducting qubits. Furthermore, an in- 
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FIG. 4. (color online) Average error per Clifford as a func- 
tion of the ZX_k/2 gate length T2. the shaded area delimits 
coherence-limited errors and is defined by the calculated er- 
ror for the measured coherence times of both qubits (dashed 
line) and the calculated error for a coherence time equal to 
twice the qubits relaxation times (dash-dotted). Error bars 
represent la confidence intervals. These results suggest that 
our average gate error is currently limited by qubit coherence. 



terleaved RB experiment in which the two-qubit gate is 
alternated with a random Clifford during the sequences 
yielded a gate fidelity of 0.9347, which compares favor- 
ably to the fidelity of 0.8799 obtained by quantum process 
tomography performed on the same gate. Our results 
show the importance of having a protocol insensitive to 
errors arising from SPAM, especially as the processes to 
benchmark become higher and higher in fidelity. Mea- 
surements of our protocol as a function of the gate du- 
ration suggest that our gate is currently limited by the 
coherence time of our qubits. 
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RANDOMIZED BENCHMARKING OF SINGLE QUBIT GATES 

We show here an implementation for our two-qubit system of the RB protocol for single qubit gates described in 
Ref. [1]. The measurements (see Fig. 1) consist of standard RB experiments on each qubit separately (C<8>I for qubit 
1, I <S> C for qubit 2) and on both qubits simultaneously (C <8> C). The former experiments gives the average error 
per single-qubit Clifford for each of the qubits, whereas the latter contains information about the amount of spurious 
crosstalk present in the system. The data in Fig. 1 were averaged over 20 random sequences for each sequence length. 

The decay of each single subsystem is fitted to the same exponential model as the two-qubit RB experiments, 
F(i) = Aa % ± B, where i is the number of Cliffords. The results are summarized on Table I, where is extracted 
from the RB experiments individually performed on qubit z, and 5a = a\ 2 — «i|2 a 2|ij with a\ 2 , ol\\2 and a 2 |i obtained 
from fitting poo Poo ±Poi and poo ±Pio, respectively, in the simultaneous RB experiments. 
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0.9918 ±0.0002 
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0.377 


«1|2 


c®c 
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0.9876 ± 0.0004 
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0.9745 ±0.0011 


0.705 


5a 




0.0002 ±0.0018 





TABLE I. Summary of the extracted data from the single-qubit RB experiments on both qubits. Uncertainties represent la 
confidence intervals. 

The average errors per Clifford n = (1 — a*)/2 obtained from the above results are r\ — 0.0041 ± 0.0001 and 
r M2 = 0.0067 ± 0.0002 for qubit 1 and r 2 = 0.0048 ± 0.0002 and r 2 \i = 0.0062 ± 0.0002 for qubit 2. The parameter 
5a represents the amount of correlations in the system, a measure of entangling errors. From our experiments we 
can see that, although there is some amount of crosstalk (5ri\ 2 = r^ 2 — r± = 0.0026 ± 0.0003 and 5r 2 \i = r 2 |i — r 2 = 
0.0014 ±0.0004) in our sample, the amount of entangling error is minimal. This reflects favourably on the performance 
of the ZX^j 2 gate, which turns out to be coherence limited and not affected by systematic errors. 
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FIG. 1. (color online) Experimental results of the single-qubit RB experiments. Qubit 1 (a) shows an average error per Clifford 
of n = 0.0041±0.0001 when measured individually (circles) and ri\ 2 = 0.0067±0.0002 when measured simultaneously with qubit 
2 (triangles). Qubit 2 (b) has an average error per Clifford of r 2 = 0.0048±0.0002 individual (circles) and r 2 \i = 0.0062±0.0002 
(triangles) simultaneous. Uncertainties represent la confidence intervals. 
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QUBIT PARAMETERS 

The relaxation and coherence times of both qubits are quoted as the average of 120 independent measurements. 
For qubit 1, the average of the relaxation time measurements was = 11.6 /is with standard deviation a^i = 1-6 
/is and the average of the coherence time measurements was =7.1 /is with standard deviation = 4.7 /is For 

(2) (2) (2) (2) 

qubit 2, Ti =9.1 /is with standard deviation cr^ = 0.9 /is and TlJ = 5.6 /is with standard deviation <r^ 2 = 0-5 //s. 

The qubits anharmonicities were (lj{ 2 - cj? 1 )/2tt = -331 MHz for qubit 1 and (u\ 2 - cu% 1 )/2>k = -216 MHz for 
qubit 2. 



DECOMPOSITION OF THE TWO-QUBIT CLIFFORD OPERATIONS 



Defining C\ as the group of single qubit Clifford operators (which has 24 different elements) , the two-qubit Clifford 
group can be found with the help of the group S\ = {/, Rs, R%} where Rs is the Pauli transfer matrix of the unitary 
operator S = exp[-z(X + Y + Z)tt/>/33]. This group is simply the rotation that exchanges all the axes of the Bloch 
sphere (the x — y — z axis maps to y — z — x). To do this we note that there are four distinct classes of the two-qubit 
Clifford group. The first class consists of 576 elements (24 2 ) and represents all single qubit Clifford operations 
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The second class has 5184 elements (24 2 x 3 2 ) and contains all combinations of the following sequence 
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We call this the CNOT-like class. The third class also has 5184 elements (24 2 x 3 2 ) and contains all combinations of 
the following sequence 



Ci 



Ci 



Si 



Si 



We call this the iSWAP-like class. It should be noted that we are using a non-standard notation for the iSWAP gate. 
The final class is the SWAP class and consists of all 576 (24 2 ) combinations of the following sequence 



Ci 



Ci 



This is the optimal decomposition of the two-qubit Clifford group in terms of number of CNOTs as it can be shown 
that implementing a iSWAP requires two CNOTs and a SWAP requires three. Thus on average the number of 
CNOTs to implement a two-qubit Clifford is 1.5. The same is also true if the building block was the iSWAP as it 
takes two of these to make a CNOT and three to make a SWAP [2]. The number of single qubit gates depends on 
how the single qubit Cliffords are implemented. It can not be less than an average of 3.8 single qubit gates for each 
two-qubit Clifford and in general it will be more. This is because, for our system, it is simpler to tune up a finite 
set of generators and decompose all Clifford operations in terms of them. For this experiment the generating set was 
{/, X ±7r / 2 ? ^±71-/25^71-, ^tt}® 2 , and ZX_ 7r / 2 - Since we can simply incorporate any single qubit operations into the pre 
C\ and any element form S\ into the post rotation we use the following replacements for the two-qubit entangling 
gates of the above classes 



zx_ 



tt/2 



zx_ 



tt/2 
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and for the SWAP class we use the replacement 



ZX_ 



tt/2 



Y-,12 















X 



tt/2 



X v/2 \-\Y. 



t/2 



tt/2 



With these decompositions we find the average number of ZX_ 7V /2 is 1.5 and the number of single qubit gates is 
either 5.6 or 7 depending on whether the identity operations are included as a single qubit rotation. This is basicially 
the same as that used in Ref. [3]. 
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